Value activation for bias alleviation: Generalized-activated deep double deterministic policy gradients

It is vital to accurately estimate the value function in Deep Reinforcement Learning (DRL) such that agent could execute proper actions instead of suboptimal ones. However, existing actor-critic methods suffer more or less from underestimation bias overestimation bias, which negatively affect their performance. In this paper, we reveal a simple but effective principle: correction benefits alleviation, where propose generalized-activated weighting operator uses any non-decreasing function, namely activation as weights for better estimation. Particularly, integrate into estimation and introduce novel algorithm, Generalized-activated Double Deterministic Policy Gradients (GD3). We theoretically show GD3 capable alleviating potential bias. interestingly find functions lead satisfying performance with no additional tricks, contribute faster convergence. Experimental results on numerous challenging continuous control tasks task-specific outperforms common baseline methods. also uncover fact fine-tuning polynomial achieves superior most tasks. Codes will be available upon publication.